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Abstract 

We propose a betting strategy based on Bayesian logistic regression modeling for the 
probability forecasting game in the framework of game-theoretic probability by Shafer and 
»^ . Vovk [16| . We prove some results concerning the strong law of large numbers in the prob- 

ed \ ability forecasting game with side information based on our strategy. We also apply our 

strategy for assessing the quality of probability forecasting by the Japan Meteorological 
Agency We find that our strategy beats the agency by exploiting its tendency of avoiding 
clear-cut forecasts. 

> ' 

' Agency, probability of precipitation, strong law of large numbers. 

cn ' 

o 1 Introduction 

In this paper we consider assessing quality of probability forecasting for binary outcomes. A 
primary example of probability forecasting is the probability of precipitation announced by 
weather forecasting agencies. The binary outcomes are either "rain" (more precisely, precipita- 
te.' tion above certain amount during a specified period at a particular location) or "no rain". In the 
United States the National Weather Service started to announce probability of precipitation in 
1965 (cf. [6}), whereas the Japan Meteorological Agency started probability forecasting in 1980 
for Tokyo area and extended it to the whole Japan in 19860. How can we assess the quality of 
probability forecasting? We propose to assess probability forecasting by setting up a hypothet- 
ical betting game against forecasting agencies in the framework of game-theoretic probability 
by Shafer and Vovk [16J. 

We can regard the capital process of a betting strategy as a test statistic of a statistical hy- 
pothesis ( [[TSl . [[13). Our null hypothesis is that given the probability pn announced by the 
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agency, the outcome is indistinguishable from the Bernoulli trial with success probability p„. If 
this hypothesis is true, then the capital process becomes a non-negative martingale and the capi- 
tal process converges to a finite value almost surely. However if the announced probability p„ is 
not good, then a clever strategy may be able to beat the forecasting agency in the betting game. 
In our game we construct a betting strategy based on Bayesian logistic regression modeling, 
which is a very standard statistical model for analyzing binary responses. We will prove some 
results on the strong law of large numbers in probability forecasting game with side information 
based on our betting strategy. We also see that our strategy works well against probability of 
precipitation announced by the Japan Meteorological Agency. 

Organization of this paper is as follows. In Section|2]we formulate the probability forecast- 
ing game with side information and derive some basic properties of betting strategies. It also 
serves as a brief introduction to game-theoretic probability theory. In Section [3] we introduce 
our betting strategy based on logistic regression model. In Section |4] we prove some properties 
of our logistic betting strategy in the framework of game-theoretic probability. In Section |5] 
we give numerical studies of our strategy. In particular we apply our strategy to the data on 
probability of precipitation announced by the Japan Meteorological Agency. We end the paper 
with some discussions in Section [6l 

2 Formulation of the probability forecasting game and sum- 
mary of preliminary results 

In this section we formulate the probability forecasting game and extend it to include side 
information. We mostly follow the results in [10]. 

At the beginning of day n (or at the end of day n - 1) an agency (we call it "Forecaster") 
announces a probability p„ of certain event in day n, such as precipitation in day n. Let x„ = 0, 1 
be the indicator variable for the event, i.e., x„ = 1 if the event occurs and Xn = otherwise. We 
suppose that a player "Reality" decides the binary outcome x„. When Forecaster announces p„, 
it also sells a ticket with the price of pn per ticket. The ticket pays one monetary unit when 
the event occurs in day n, i.e., the value of the ticket at the end of day n is x„. A bettor or 
gambler, called "Skeptic", buys M„ tickets with the price of p„ per ticket. Then the payoff to 
Skeptic in day n is M„(Xf, - p„). We allow M„ to be negative, so that Skeptic can bet also on 
the non-occurrence of the event. If the probability announced by the agency is appropriate, it is 
hard for Skeptic to make money in this game. On the other hand, if the probability is biased in 
some way. Skeptic may be able to increase his capital denoted by Hence we can measure 
the quality of probability forecasting in terms of TC. 

We now give a protocol of the game, following the notational convention of lfT6l . 

Binary Probability Forecasting (BPF) 
Protocol: 

Skeptic announces his initial capital TCo = 1. 
FORn = 1,2,...: 

Forecaster announces Pn e (0, 1). 
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Skeptic announces M„ e R. 
Reality announces x„ e {0, 1}. 

Collateral Duty: Skeptic must keep 7C„ non-negative. 



Forecaster is supposed to decide its forecast pn based on all relevant side information avail- 
able at the time of announcement. We modify the above protocol so that Forecaster also dis- 
closes the relevant side information c„, which is a <i-dimensional column vector, together with 
the probability p„. Furthermore we define auxiliary capital processes S„ and 'V„. 

Binary Probability Forecasting With Side Information (BPFSI) 
Protocol: 

7^:= l,<So:=0,%:=0. 
FORn = 1,2,...: 

Forecaster announces p„ 6 (0, 1) and c„ 6 R''. 
Skeptic announces M„. 
Reality announces x„ e {0, 1}. 

Sn := <5„_i -I- C„(X„ — Pn)- 

'Vn ■= "Vn-l + CnC'„ Pn{l - Pn)- 

Collateral Duty: Skeptic must keep TCi non-negative. 

In the protocol, c'^ denotes the transpose of c„, 7C„ is a scalar, <S„ is a J-dimensional column 
vector and 'Vn is ad x d symmetric matrix. 

If J = 1 and c„ = 1, then S„ = Tj'Lii^i ~ Pd- When we study the usual strong law of 
large numbers in game-theoretic probability, we are interested in the convergence Sn/n 
as n oo. Generalizing this case, in the presence of side information, we are interested in 
the convergence 'V'^Sn — > 0, although the order of 'V„ may be different from 0{n)- We call 
this convergence the usual form of the strong law of large numbers in BPFSI. See Theorem 
14. II in Section BTTl However, as we prove in Theorem |4.2| of Section under mild regularity 
conditions, we can prove a stronger result 

lim^(^„)-'.S„ = 0, 

n 

where gi'V) is close to "V^'^ such as gi'V) = 6 > 0. 

Let 



v„ = 



denote the fraction of the capital Skeptic bets on day n. Then the capital process 7C is written 



as 



n<n = \\{\^viXi-p>i). (1) 



i=l 
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Now suppose that Skeptic himself models Reality's move as a Bernoulli variable with the 
success probability Pn e (0, 1). If Skeptic totally trusts Forecaster, then he sets Pn = Pn- How- 
ever if Skeptic does not totally trust Forecaster he may formulate pn differently from p„. Fur- 
thermore suppose that Skeptic uses the "Kelly criterion" ( [iT2l . [|9l) to determine v„ so as to 
maximize the expected value of the logarithm of the capital growth under p„: 

v„ : EpX\o%{l +v{Xn- Pn))] ^ Hiax . 

Writing 

£'^Jl0g(l + v(Xn - Pn))] = Pn log(l + v(l - Pn)) + (I - Pn) l0g(l - VPn) 

and differentiating this with respect to v, the unique maximizer v„ is obtained as 

Pn - Pn Pn ^ ' Pn 
= —p. T = -j ■ (2) 

Pn{^ - Pn) Pn ^ - Pn 



With this choice of v„ we have 

1 -I- Vn(Xn - Pn) 



Hence ([T]) is written as 



'Kn = 



[PnlPn if ^„ = 1 

'\{l-Pn)l{l-Pn) ifx„ = 

' Pn(l-Pn)'-'"' 

n"=i^f(i-Ay-^' 



n-=ipf(i-A)'-'^'' 

In the case that Skeptic models the joint probability p(xi, Xn) of Reality's moves, p„ is 
given as the conditional probability 

^ _ p(jCi, ■ ■ ■ , Xn-l, 1) 
p(Xi, . . . , Xn-l) 

In this case 

\l—x„ P^^l') • • • 1 ^n-\t ^n) 



and %„ is written as 



mi - PnY-" = — = 0, 1, 

p{X\, . . ., Xn-l) 



p{X\, . . . , Xn) 



^"^n-=i;^f(i-A-V-- 

For the rest of this section we introduce some terminology of game-theoretic probability. 
An infinite sequence of Forecaster's moves and Reality's moves 

^ = P1C1X1P2C2X2 . . . 
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is called a path. The set Q. of all paths is called the sample space. A subset £ c Q is an event. 
A strategy V of Skeptic determines p„ based on a partial path pxC\X\... pn-\Cn-\x.n-\Prfn'- 

V : P\C\X\ . . . Pn-\C„-iX„-iP„C„ Pn, n = 1,2, 

'K^ = 'K^iO denotes the capital process when Skeptic adopts the strategy P. We say that 
Skeptic can weakly force an event £ by a strategy P if 'K^ is never negative and 

limsup-TCC^) = oo \/^^E. 

n 

For two events Ei,E2 c Q., E'^ U E2 is denoted as Ei => E2, where E^ is the complement of 
El . We say that by a strategy P Skeptic can weakly force a conditional event Ei => E2 if TC^ is 
never negative and 

lim sup TCf (^) = °° e £1 n 

£■1 is interpreted as a set of regularity conditions for the event E2 to hold. 

Let /Imax.n ^ud /Imin.n dcuotc the maximum and the minimum eigenvalues of 'V„. In this paper 
we consider the following regularity conditions: 

i) lim„ Airm.n = 

ii) limsup„ 

iii) {ci, C2, . . . } is a bounded set. 
Namely we take Ei as 

£■1 = I lim/lmin,n = ^, Um sup /lniax,n /^min.n < ^ and Cj, C2, ... are bounded}. (4) 

" n 

The condition i) makes the meaning of ""Vn — » oo" precise. The condition ii) means that 'Vn 
stays away from being singular. For J = 1 ii) is trivial and not needed. 



3 Logistic betting strategy 

In this section we introduce a betting strategy based on logistic modeling of Reality's moves. 

As in the previous section Skeptic models x„ as a Bernoulli variable with the success proba- 
bility Pn. Furthermore we specify that Skeptic uses the following logistic regression model for 
the logarithm of the odds ratio: 

log--^=log--^ + 0'c„, (5) 

where e is a parameter vector 

In previous studies in game-theoretic probability, many strategies of Skeptic depend only 
on Xi - Pi, i < n - I, and do not depend on p„. However obviously it is more reasonable to 
consider Skeptic's strategies which depend on Strategies explicitly depending on p„ are also 
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important from the viewpoint of defensive forecasting ( [|20ll . IfTSlI '). We again discuss this point 
in Section 1331 

We now consider the capital process "TC^ of Q for a fixed 6 e Mf'. Solving for p„ we have 
Then 

0' c X 

P:-il-Pn?-'"=P:;'(l-Pn?-''" ' 



1 + pnie^''-" - 1) 
and the capital process is written as 



-TC^ = till L!i — = . (7) 

" \j Pf(^ - Pif-'- n;ii(l + Pie'''- - D) ^ ^ 

Naturally it is better for Skeptic to choose the value of 6 depending on the moves of other 
players. In this paper we consider a Bayesian strategy, which specifies a prior distribution 
n{9) for 6. Bayesian strategies for Binary Probability Forecasting with constant p„ = p was 
considered in [,10 J . Bayesian strategy is basically the same as the universal portfolio by Cover 
and his coworkers ([|3l, flU, [|51). In the universal portfolio, a prior is put on the betting ratio 
V itself, where as we put a prior on the parameter of Skeptic's model. Furthermore differently 
from we allow continuous side information. 

In the Bayesian logistic strategy with the prior density function n(6) of 0, the capital process 



is written as 



jRd Jr</ ri,=i(l + Piie'''- - 1)) 

TC^ is of the form (|3]) where 

...,.„)= n .,'.(1 - p,)-. I _ 



(e)de. 



In this paper we consider a prior density which is positive in a neighborhood of the origin. 
We call such n "a prior supporting a neighborhood of the origin". 

4 Properties of logistic betting strategy from the viewpoint of 
game-theoretic probabihty 

In this section we prove game-theoretic properties of our Bayesian logistic strategy. 
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4.1 Weak forcing of the usual form of the strong law of large numbers 

The first theoretical resuk on our logistic betting strategy is the following theorem. 

Theorem 4.1. In BPFSI, by a Bayesian logistic strategy with a prior supporting a neighborhood 
of the origin, Skeptic can weakly force 

E: ^ lim'V-^Sn = 0, 

n 

where Ey is given in 

The rest of this subsection is devoted to a proof of this theorem. The basic logic of our proof 
is the same as in Section 3.2 of fl^. 

We first consider the logarithm of TC,^ in (|7]) for a fixed 6: 



log-TC = 0'Yj - Z + P'^^''^' - 



i=l i=\ 

For notational simplicity we write 

u{e) = \ogX- 

We investigate the behavior of u{6) for 6 close the origin. Fix 6 e with unit length (i.e. 
1 1^1 1 = 1) and consider u{s6), < s < e. Note that m(0) = 0. We will choose e appropriately later 
in([II]). 

The derivative of u{sff) with respect to s is written as follows. 



-uise) = e^c,x,-^——-^ 



,=1 ' P'^' 

^'V . ^ X^e'dPie^''^- -ffc^piil+p^ie^'''- - D) 

= e ) Ciixi - Pi) - ) — 

Z_J Z_J 1 + _ 1) 

= e'Sn - > e'ciPiii - Pi) . (8) 

Note that 6'ci and e^^''^' - 1 have the same sign and hence each summand in the second term on 
the right-hand side of ([8]) is non-negative. 
Let 

1 -I- p{e^ - 1) 

be a function of y e R depending on the parameter p e [0, 1]. Note 7p(0) = 0. Its derivative is 
computed as 

' (1 -I- p{e^ - l))^ 

Hence 
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where for negative y < we interpret £(.■■■ )dz as - ^^{- ■ ■ )dz. Now y'p{z) in ® is monotone 
in p with -/^(z) = and y[{z) = Hence 

= min(e"~, r') < y'p{z) < max(e"^, = e'^'. 

Then for z between and y we have 

e"'-"' < r;(z) < e'^l. (10) 
Using the upper bound e'^' and integrating y'p{z) we obtain 

\ypiy)\ = , , <lyk'^'' and 0<yy;.(j)=y . </el^l. 

1 + p{e^ - 1) 1 + p{e^ - 1) 

Let Lc,„ = maxi<j<„ ||c,||. Then 

Q n 

—uisO) > e'Sn -sYj^e'cifpiii - pde'"^'" = e'Sn - se'^nee'"^- 

!=1 

and integrating this for < 5 < e we have (for any Q and 6 > 0) 

u{ee) > ee'Sn - —e'^^nOe'^". 

For the rest of our proof we arbitrary choose and fix a path ^ e Ei, where Ei is given in 
dU. Various constants (e's, L's etc.) below may depend on ^. By iii) there exists such that 
Lf. „ < for all n. Also there exist hq and such that /^max,n/^min,n < for all n > hq. Now 
suppose that "V'^Sn for this ^. Then for some e\ > and for infinitely many n we have 
||'V~^<S„|| > 6i. Let Ni = {ni,n2, . . . } be a subsequence such that ||'V;^'<S„|| > e\ for n e M\. The 
normalized vectors 

Tin = — —, — —, neNi, 

have an accumulation point 77, ||?7|| = 1, and hence along a further subsequence N2 c Ni we have 

lim r]n = 77. 

By Cauchy-Schwarz, for three vectors a,b,c e 'R'^, we have 

|Z7'%C| ^ in,ax,„ll^lll|c|| ^ ||&||||c|| ^ ^ 

^— — < — — < L^^^, Vn>no- 

^ ' ft^^ ''^min /7 1 1^1 1 1 1^1 1 

Then we can choose < €2 < 1/4 such that for all sufficiently large n e N2 and for all 77, 
||?7|| = 1, sufficiently close to rj, we have 

u(eii) > efj'Sn - —fj"V„fje'^'-" 

= e\\^;,'Sn\W^nr]n ' y ^^'^'Z^^'^^" 

> 66i77'%77(l - 62) - y^'%/7(l + f2)e^^^ 

= e77'%77 (6i(l-62)-|(l+62)e^^0- 
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We now choose small enough e > such that 

6l(l-62)-|(l+62K^^>|. (11) 

Then 

Uief]) > -Y^min,n °° (/l ^ OO, n G N2). 

Note that the convergence is uniform for fj in some neighborhood N{ri) of ?]. Since our prior n 
puts a positive weight to N{erj), 'K" oo along n e N2. This completes our proof of Theorem 

EH 



4.2 Weak forcing of a more precise form of the strong law of large num- 
bers 

As discussed in Section [21 we can establish a much more precise rate of convergence of the 
strong law of large numbers based on our Bayesian logistic strategy. Our main theorem of this 
paper is stated as follows. 

Theorem 4.2. In BPFSI, by a Bayesian logistic strategy with a prior distribution supporting a 
neighborhood of the origin, Skeptic can weakly force 

E, ^ limsup , <1, 
„ ^logdet^„ 

where Ei is given in 

We give a proof of this theorem in the following three subsections. 



4.2.1 Bounding the maximum likelihood estimate 

We now consider the behavior of TC,^ in (|7]), when 'K^ is maximized with respect to 6. Let 

ei = argmaxTC^ 

We call 61 the maximum likelihood estimate, since is of the form of the likelihood function 
of the logistic regression model. It is easily seen that the maximizer 6* is finite except for a 
special case that the vectors in {c, | x, = 1} U {-c, | Xi = 0} lie on a half-space defined by a 
hyperplane containing the origin. More specifically in Lemma |43] we prove that \\6*\\ is small 
when ||'y„ 'iS„|| is small. 

The maximizing 0* can only be computed at the end of day n after seeing all the data 
Pi,ci, xi, . . . , pn, Cn, Xn- Hcncc wc Call a strategy using 0* a "hindsight strategy", which is the 
same as the best constant rebalanced portfolio (BCRP) in the terminology of the universal port- 
folio. 

We prove the following lemma. 



9 



Lemma 4.3. Let L^^n = maxi<,<„ ||c,|| andLx^n = ^max,nl ^mm,n, where we assume /lmin,« > 0. Then 

||%-i<S„|| < — ^ ^ ll§:il < 3L,,„||%-i5„||. 

For any fixed ^ e Ei, there exist Lc,La, such that Lc,„ < Lc and < for all sufficiently 
large n. Also in Theorem 14. II we proved that Skeptic can weakly force Ei => lim„ 'V„^S„ = 0. 
From these results we have the following proposition. 

Proposition 4.4. In the same setting as in Theorem \4.1\ Skeptic can weakly force E\ => 
lim„ ei = 0. 

The rest of this subsection is devoted to a proof of Lemma 1431 Consider the inner product 
O'VuiO) = 9' grad uiO) of 6 and the gradient of uiff). If O'Vuiff) < 0, then the gradient points 
toward the interior of the ball with radius r = \\6\\ as shown in Figured] If 6'Vu{6) < for all 
6 with ll^ll = r, then \\6*\\ < r. This can be seen as follows. Suppose \\6*J\ > r. Let 6 be the 




Figure 1 : Gradient of u(9) 



maximizer of u(6) on the sphere (the boundary of the ball). Then at 6 the gradient of Vm(^) is a 
positive multiple of 9 and this contradicts 9'Vu(6) < 0. 

As in the previous subsection, using this time the lower bound in (flOl ). we have 



Now 
and 
Hence 
Furthermore 



\e's„\ = \e'^„^„'s„\ < \\e'^„\\ ■ \\^;'Sn\ 



e'^nOe 



-Lc,„\\ 
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Therefore 

For\\e\\ = 3UJ^;'S„\\ 

Then for ||%i<S„|| < 1/(3L,,„L,,„) 

3g-3Z.,,.„i.,„ll%7'5„ll > 3g-l > 

Hence, if ||%i<S„|| < 1/(3L,,„L^,„), we have e'Vu{9) < for all 9 with ||0|| = SL^J'V^^SnW. By 
the remark just after Proposition 14.41 this completes the proof of Lemma B31 

4.2.2 Behavior of the hindsight strategy 

We summarize properties of log 7C„" in view of the standard theory of exponential families (L2J) 
in statistical inference. Define 

n 

ud) = logd + Piie''- - D), m = Yj 

Note that is the cumulant generating function (potential function) for the logistic regres- 
sion model, which is an exponential family model with the natural parameter 9. Hence each 
il/i{9) and i]/{9) are convex in 9. Indeed by (0), the Hessian matrix H^.(9) of i/r,- is given as 



'(1+aV^'-1))2' 
which is non-negative definite. The Hessian matrix 



H^(9) = J]h^,(0) 



of if/ is positive definite if 'Vn is positive definite, which is the Fisher information matrix in terms 
of the natural parameter 9. 

Convexity of i/^,- implies concavity of log TCjf = 9'T„ - iff(9), where 

n n 

T~n — Ci^i — Sn + ^"'^ CjPi. 

i=\ i=\ 

Hence if the maximum of log'TC,^ is attained at a finite value §*, then the "maximum likelihood 
estimate" satisfies "the likelihood equation" 
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or equivalently 

Tn = V^(§:). (12) 
The likelihood equation can also be written as 



,=1 1 + Piie'^"'- - 1) 

From this it follows that = if and only if 7~„ = ^tPt- 

Regard (fT2l) as determining in terms of t = 7~„, i.e., = 6*Jj:), t = 7~„. This is the inverse 
map oit = Vtff{9). Differentiating t = Viff{9) again with respect to 9 we obtain the Jacobi matrix 

dt 

J = — = HA6) 
89 ^ 

as the Hessian matrix of i/^. Hence the Jacobi matrix 891/ dT,, written as 

dt 



H^mr,))-'. (13) 



Now logTCn" = log'K/ is the Legendre transformation (cf. Chapter 3 of LU) of log TC^: 



log7C,P^ = §:(0'?-<AtoX t = T„. 



Differentiating log TC,^"*^'^ with respect to t, by (fT2)) we obtain 

- log = Kit) + {t- Vip{9l{t))^ = 9:(t). (14) 
8t 8t 

By ^ the Hessian matrix of logTC,^'^ is given by H^{9l{t))-\ 
We are now ready to prove the following proposition. 

Proposition 4.5. With the same setting as in Lemma \4.3\ 
where C„ = 3Lc,„L^ „. 

Proof. For given 7~„, To = YIUi CiPi and for s e [0, 1], consider 
Then logTcf"^^"^ = ^(1)- It is easily seen that ^(0) = 0. By Gl) 

g'is) = e:ito + sSnYSn. 
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Again it is easily seen that g'{0) = 0, since §*(7~o) = 0. Then 



Now 



^1)= r r g"(u)duds. 

Jo Jo 

g"(u) = s'.H^eiito + uSn)y'Sn. 



Also Iduds = 1/2. Furthermore by LemmaO if ll^,;^<S„|| < 1/(3L,,„L^,„) then \\ei(To + 
uSf,)\\ < 3L^ „||'y^'iS„|| for all < M < 1. Combining these results we have the proposition. □ 

As in Proposition |43] we have the following corollary. 

Corollary 4.6. In the same setting as in Theorem \4. 1 1 Skeptic can weakly force 

logTC? 

=> lim 



4.2.3 Laplace method for evaluating the difference of the hindsight strategy and the lo- 
gistic strategy 

In the last subsection we clarified the behavior of the capital process for the hindsight strategy. 
Now we employ the standard Laplace method to evaluate the diff"erence of the hindsight strategy 
and the logistic strategy (Section 5 of LIJ, Chapter 3.1 of [,8 J). 

Lemma 4.7. Let nbe a prior density supporting a neighborhood of the origin and let denote 
its capital process. For ^ & such that lim„ 'V^^Sn = 0, 

(l/2)logde.T„ <'5> 

Proof. For 6 close to the origin, expanding log TCf around 6* we have 

logTC,^ = iog<" - ^(e - eiYH^iOnXe - ei), 

where is on the line segment joining 6 and §*. Hence 

< = <" X exp(-^(e - ei)'H^{en){e - ei)). 

Now by the standard Laplace method we obtain (fTSl) . □ 
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Finally we give a proof of Theorem 14.21 
Proof of Theorem \4.2\ By Corollary 14.61 and Lemma 14.71 

Hence if lim sup„ S'/V~^Snl log det "y,, > 1, then lim sup„ log '7Q = oo. □ 
4.3 Monotonicity with respect to the forecast probability 

Here we consider the case that log(p„/(l - p„)) itself is an element of the vector of the side 
information c„ and hence is multiplied by a coefficient in (|5]). For notational convenience we 
here eliminate log(/>„/(l - />„)) from c„ and write (|5]) as 

log --^ = ySlog T-^ + T„, (16) 

^-Pn 1 - Pn 

where t„ denotes the effect of side information other than log(p„/(l - p„)). Intuitively jS rep- 
resents how much trust Skeptic puts in Forecaster. If yS = then Skeptic entirely ignores Fore- 
caster's p„ and if yS = 1 then Skeptic takes p„ for granted. The value of jS e (0, 1) corresponds 
to partial trust in p„. It is somewhat surprising to see that yS > 1 in the case of probability of 
precipitation announced by the Japan Meteorological Agency in Section [521 

We now investigate how v„ in Q behaves with respect to p„ forgiven pi,ci,X\,. . . ,p„-\,Cn-\, 
x„-\ . This is an important question from the viewpoint of defensive forecasting ( [|20l . IfTSl ). be- 
cause in defensive forecasting we want to obtain p„ for which y„ = 0. For notational simplicity 
we now omit the subscript n and write <^ as 



p-p p^^^e^ -{l- pf-^ 



Then 

v{p) = 



p{\-p) pPe^ + {l-pf 
Differentiating this with respect to p we obtain 

dv{p) _ -e2^p2(/j-i) + - pf'^ifi - 2 + 2p(l - p)) - (1 - pfl^-^ 

dp ~ {pl^e^ + (1 - pff ' 

The numerator of dv{p)/dp can be written as 

-(eV' - (1 - pf'V + e'(fi - 1)/"^(1 - pf~\ 
which is non-positive for /3 < I. Hence we have the following proposition. 
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Proposition 4.8. Under the logistic regression model (1161) . for /3 < I the betting ratio Vn(Pn) is 
monotone decreasing in p„. 

It is natural that v„ is monotone decreasing in p„, because if />„ is too high and Skeptic does 
not believe it, then Skeptic will bet on the non-occurrence x„ = 0. 
For the special case of /3 = 1, 

e^" - 1 

VniPn) = 



1+Pnie^" - 1)' 

which is bounded and monotone in />„ e [0, 1]. For yS < 1, v„(/>„) is unbounded and it can be 
easily seen that 

lim = 1 , lim = - 1 . 

P„10 l/pn Pnn 1/(1 - p„) 

We can interpret the first limit as follows. Suppose that p„ = 1/1000, i.e. the price of a ticket is 
1/1000 of a dollar. In this case Skeptic can buy 1000 tickets with one dollar and has the chance 
of winning 1000 dollars. Hence Skeptic may want to buy 1000 tickets. Thus it is reasonable 
that V and p„ are inversely proportional when p„ is small. 



5 Experiments 

In this section we give some numerical studies of our strategy. In Section [STTI we present some 
simulation results and in Section 15.21 we apply our strategy against probability forecasting by 
the Japan Meteorological Agency. 

5.1 Some simulation studies 

We consider three cases and apply three strategies to these examples. In our simulation studies 
Reality chooses her moves probabilistically, either by Bernoulli trials or by a Markov chain 
model. 

• Case 1 : x„ is a Bernoulli variable with the success probability 0.7 and pn alternates be- 
tween 0.4 and 0.6 (i.e. OA = pi = p^ = ■ ■ ■ and 0.6 = p2 = P4 = • • •)■ 

• Case 2: Xn is a Bernoulli variable with the success probability 0.5 and p„ alternates be- 
tween 0.4 and 0.6. 

• Case 3: p„ = 0.5 and x,, is generated by a Markov chain model with transition probabili- 
ties shown in Figure |2] 

• Strategy 1: 6* is a scalar and c„ = 1 in Q. Assume that the prior density for 6 is given as 
uniform distribution for [0,1]. The capital process is written as 

" Jo nUl+Pi(e'-l)) 
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0.7 




0.7 



Figure 2: Transition probabilities for x„ 



• Strategy 2: 6' = [6i,fi - 1] and = [l,log -^]- Assume independent priors for 6i and 
/3, which are uniform distributions over [0,1]. The capital process is written as 



Jo Jo Yll^ 



dOidfi. 



• Strategy 3: 6' = [6i,/3- 1,63] and = [l,\og-^^,Xn-\]. Assume independent priors for 
61, /3 and 9s, which are uniform distributions over [0,1]. The capital process is written as 



1 /-I ^1 E;Li-v,+(y8-i)i:;Li-^. log 1^+63 2;l,-v,^.-i 

^0 ^0 n"=i(i+A-(e''^*^"'^""'^^'''" -1)) 



dOidfidOs. 





100 200 



100 200 300 400 



Figure 3: Case 1 with strategy 1 



Figure 4: Case 2 with strategy 1 



As shown in Figure |3] and Figure HI we can beat Reality by strategy 1 only in case 1. So we 
improve our strategy and apply strategy 2 to case 2. We can see from Figure |5] and Figure |6]that 
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Figure 5: Case 2 with strategy 2 



Figure 6: Case 3 with strategy 2 



10^° 




10 ' ' ' ' ' ' 

100 200 300 400 500 

Figure 7: Case 3 with strategy 3 



strategy 2 can work well in case 2 but still not effective in case 3. Finally, we use strategy 3 in 
case 3 and observe that it shows a good result for Skeptic in Figure |71 

From these simulations, we see that Skeptic can beat Reality with more flexible strategy 
utilizing more side information. 

5.2 Betting against probability of precipitation by the Japan Meteorolog- 
ical Agency 

Now we apply our strategy to probability of precipitation provided by the Japan Meteorological 
Agency. We collected the forecast probabilities for the Tokyo area from archives of the morning 
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edition of the Mainichi Daily News and the actual weather data on 9:00 and 15:00 of each 
day for Tokyo area from|http: //www. weather-eye jXom/lfor the period of three years from 
January 1 , 2009 to December 3 1 , 201 1 . We counted a day as rainy if the data on this site records 
rain on 9:00 or on 15:00 of that day in Tokyo area. 

The forecast probability p„ is only announced as multiples of 10% (i.e. 0%, 10%, . . . , 90%, 
100%) by JMA. The data are summarized in Table [T] p„ represents the probability of precipita- 
tion on day n and Xn indicates the actual precipitation. Actual ratio is calculated from the ratio 
of the number of rainy days to all days for a given value of p„. 



Table 1 : Actual ratio of rainy days 



Pn(%) 


Xfi — \ 


Xn = 


Actual Ratio(%) 





1 


61 


1.6 


10 


10 


324 


3.0 


20 


24 


193 


11.1 


30 


36 


117 


23.5 


40 


20 


26 


43.5 


50 


67 


56 


54.5 


60 


38 


14 


73.1 


70 


36 


7 


85.7 


80 


36 


4 


90.0 


90 


22 


1 


95.6 


100 


3 





100 



The distinct feature of the prediction by JMA is that that p„ tends to be closer to 50% 
than the actual ratio. For example, when JMA announces p„ = 20%, the actual ratio is only 
11.1%. Similarly when JMA announces pn = 80%, the actual ratio is 90%. Hence JMA has the 
tendency of avoiding clear-cut forecasts. 

In the hindsight strategy, the value of /3, which is a coefficient for log(p„/(l -p,,)) in strategy 
3 is close to 1.5. Hence we modified strategy 3 of the previous section, so that the prior for /3 
is uniform between and 2. We also substituted />„ = 1% and p„ = 99% for p„ = 0% and 
Pn = 100%, respectively, because our strategy is not defined for pn = 0% or 100%. Figure |S] 
shows the behavior of strategy 3 and the approximation iS','y~'<S„/2. We see that our strategy 
works very well against JMA by exploiting its tendency of avoiding clear-cut forecasts. It is 
also of interest that the capital process shows a seasonal fluctuation and it does not perform well 
for the rainy season (June and July) in Tokyo area. 

6 Summary and discussion 

In this paper we proposed a Bayesian logistic betting strategy in the binary probability fore- 
casting game with side information (BPFSI). We proved some theoretical results and showed 
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Figure 8: Beating JMA by strategy 3 with yS uniform over [0, 2] 



good performance of our strategy against probability forecasting by Japanese Meteorological 
Agency. 

Here we discuss some topics for further investigation. 

We considered implications of a single Bayesian logistic betting strategy in BPFSI. We 
can also take a look at the sequential optimizing strategy (SOS) of [11 J in BPFSI. Under the 
condition 0* — > 0, Bayesian strategy and SOS should behave in the same way. However we 
could not succeed to prove weak forcing of ^ by SOS alone. 

For the case of <i = 1 we could employ approaches of [14] to prove results similar to The- 
orem |4]2l In [14J we also discussed Reality's strategies. It is of interest to study strategies of 
Forecaster or Reality in the binary probability forecasting game with side information. Defen- 
sive forecasting ( [|20l . [[T8l ) can be considered as a strategy of Forecaster. 

We extended the binary probability forecasting game by including side information. In our 
formulation side information c„ is announced by Forecaster and in our logistic betting strategy 
Cn is used as regressors in a logistic regression. However Skeptic can use any transformation 
of c„ in his strategy. In this sense, it might be more natural to formulate the game, where 
Cn is announced by Skeptic. Binary probability forecasting game is often considered from the 
viewpoint of prequential probability ([7J) and leads to the notion of randomness of the sequence 
P1X1P2X2 . ■ . ( [191 , f\3l ). From the viewpoint of prequential probability it might also be natural 
to consider side information c„ as a part of moves by Skeptic for testing the randomness of 
P1X1P2X2.... 

We assumed multidimensional c„. However from the viewpoint of game-theoretic probabil- 
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ity, we do not lose much generality by restricting c„ to be a scalar, since if Skeptic can weakly 
force events Ei,...,E4 then he can weakly force Ei D • • • D E^. By the same reasoning we can 
also consider d = oo, because if Skeptic can weakly force Ei, E2, . . ., then he can weakly force 
ri°lyEi. Interpretation and formulation of side information in game-theoretic probability needs 
further investigation. 
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